Image Inpainting with Gaussian Processes

This project investigates the applicability of Gaussian processes (GPs) as a prediction method for image inpainting, a process which reconstructs lost or deteriorated parts of an image based on the remaining portion. The use of GPs in this context not only allows us to make a single prediction for the missing region but also to draw multiple samples consistent with the context. Also, the covariance function of the GP can be adapted to the local image structure, to improve results. The main goal is to analyse the circumstances and extent to which the GP regressionmodel of Williams and Rasmussen (1996) can infer the missing parts of an image conditioned on its known parts. To do this, variations of the Squared-Exponential (SE) kernel are used, among which a new rotational-ARD SE kernel [1] is introduced to adapt to the inherent rotation of the local image structure. For evaluation, one randomly positioned patch of size 30×30 pixels is removed from each of 100 grayscale images with generic-themed content, and then manually categorised into image sets based on the arrangement of textures that compose the original content of each patch. We evaluate the relative performance of the different SE kernels on each of the image sets through a SMSE-based metric. A linear regression (LR) model is used as a simple baseline for comparison. The analysis shows that the GP regression model is effective for simple local image structures composed of low-frequency textures and edges with similar rotations.